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REMARKS 

Claims 1-18 remain pending in this application for which applicant seeks reconsideration. 

Amendment 

Claims 1-16 and 18 have been amended to clarify that the present invention is directed to 
an image search apparatus/method. That is, the superfluous language "management and" has 
been deleted to remove any ambiguity arising from using this language. In this regard, the 
specification has been amended accordingly. A substitute specification and a marked up version 
showing the changes are attached. The claims have been amended to clarify that the rearranging 
means/step rearranges images according to importance of the keywords in relation to the content 
of the images being searched. The claims further have been amended to remove all references to 
a storage means/step to broaden the claims. Claims 13-16 and 18 have been further amended to 
improve their form, using proper language, "instructions" rather than "method" in referring to 
what the storage medium holds. No new matter has been introduced. 

Art Rejection 

Claims 1-18 were rejected under 35 U.S.C. § 102(b) as anticipated by Sano (USP 
5,038,379). Applicant traverses this rejection because Sano would not have disclosed or taught 
rearranging the searched images according to the importance of the keywords in relation to the 
image content. 

Claim 1 calls for search means for searching for keywords corresponding to images to be 
searched and rearranging means for rearranging images corresponding to the keywords searched 
by the search means according to importance of the keywords in relation to the content of the 
images being searched. Other independent claims recite similar or parallel features. According 
to the present invention, keywords are assigned to image data correspondingly to the importance 
based on the content of the images being searched, and the search results (images) are arranged 
according to the importance based on the content of the images in relation to the search keyword. 
Note that the importance is not assigned to the keywords themselves, but rather to the relation 
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between the keyword and the content of the images. For example, a keyword "apple" can have a 
different degree of importance, depending on how the images depict an apple. If an image 
predominantly depicts an apple, a higher degree of importance is assigned to that image than 
another image depicting an apple in a minor fashion. Sano does not disclose or teach such a 
feature. 

Specifically, Sano discloses an image information searching apparatus that converts a 
scanned image to a character recognized by OCR, stores the character code corresponding to the 
image into a database, and displays the document image corresponding to the character resulting 
from a keyword search thereof. See column 5, lines 5-21. Sano does not teach assigning 
importance of the keyword, let alone in relation to the image content. In Sano, the image data 
assigned to an input keyword is output so that the user can search a smaller set (discriminated 
images) from the images stored in its database. The greater the number of images the database 
contains, the greater the number of images output. Sano does not disclose rearranging the output 
images according to the assigned importance. The passage (column 2, lines 58, et seq.) identified 
by the examiner merely states that assigning a keyword to an image discriminates the images that 
are assigned with the keyword from the images that are not assigned with the same keyword. In 
the present invention, the already discriminated images are rearranged based on their importance. 

Moreover, independent claims 3, 9, and 14, call for using a synonym index to find terms 
of the keyword that represent the same or similar meaning ("kewords closely related in 
meaning"). Sano further does not disclose or teach this feature. Indeed, Sano discloses using a 
dictionary having pattern data of characters for checking text during OCR where an image is 
being converted to text. This is quite different from using a synonym-based index to search for a 
keyword stored in the database that is different from an input search keyword. These claims 
further call for assigning priority to an image in relation to both the importance and the 
accordance. Sano further does not disclose such features. 
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Conclusion 

Applicant submits that claims 1-18 parentally distinguishes over the applied references 
and is in condition for allowance. Should the examiner have any issues concerning this reply or 
any other outstanding issues remaining in this application, applicant urges the examiner to 
contact the undersigned to expedite prosecution. 



Respectfully submitted, 




LYLE K 1 MMS 

/ Reb. No. 34079 

Date: June 24. 2004 ^7frlg /C — — 

Marc A. Rossi 
Registration No. 31,923 

ROSSI & ASSOCIATES 
P.O. Box 826 
Ashburn, VA 20146-0826 
Phone: 703-726-6020 
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Our Proposed Amended Claims 



1. (Currently amended) An image search apparatus-r 
which searches imago data according to keywords stored in 
relation to said image data, said image search apparatus 
comprising: 

input means for inputting a search query; 

storage means for storing a keyword table, wherein 
the keyword table includes a keyword that corresponds to 
a plurality of images, and a plurality of levels of 
importance of the keyword with respect to said plurality 
of images; ^ — 

search means for searching said plurality of image 
data images according to fcfee an input search query related 
to the keyword inputted by said input means and the keywords 
stored in relation to said image data ; 

acquiring means for acquiring said plurality of 
levels of importance of the keywords keyword , the 
importance being stored in relation to - the keywords stored 
in relation to said image data based on the images searched 
by said search means; and 

output means for outputting tke said plurality of 
image data images searched by said search means in an order 
according to said plurality of levels of importance of the 
keywords keyword acquired by said acquiring means. 

2. (Currently amended) An image search apparatus 
according to claim 1, wherein said output means outputs 
^fehe said plurality of image data images according to a 
standard level of importance of a default when said 
plurality of levels of importance is are not stored in 
relation to the keywords keyword . 

3. (Currently amended) An image search apparatus 
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according to claim 1 , which searches image data according 
to keywords stored in relation to said image - data; — said 
image search apparatus comprising: 

storage means storing the keywords stored in 
relation to said imago data/ — synonyms of said keywords/ 
and accordance between said keywords and said synonyms; 

input means for inputting a search query; 
— search means for searching said image data according 
to the search query inputted by said input means/ the 
keywords stored in relation to oaid image data/ and the 
synonyms stored by said storage means; 

— first acquiring means for acquiring importance of 
the keywords/ the importance being stored in relation to 
the keywords stored in relation to said image data searched 
by said search moans; 

wherein second said acquiring means for acquiring 
acquires said a level of accordance between at least one 
synonym of said keyword and said input search query when 
oaid search means has searched said image data according 
to said synonyms; and 

said output means for outputting outputs the said 
plurality of image data images searched by said search means 
in an order according to said plurality of levels of 
importance of the keywords keyword acquired by said first 
acquiring means and said level of accordance acquired by 
said second acquiring means. 

4. (Cancelled) 

5. (Cancelled) 

6. (Currently amended) An image search apparatus 
according to any of claims claim 1 through 3 , wherein said 
image search apparatus is provided in a server of an 
information retrieval system on the Internet/ oaid an input 
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means receiving a search query inputted from a WWW browser 
of a client of said information retrieval system on the 
Internet, said output means outputting the said plurality 
of image data images searched by said search means in a 
predetermined format, which includes HTML and XML, to said 
client in said order . 

7. (Currently amended) An image search method 
applied to an image search apparatus , which searches image 
data according to koywordo stored in relation to oaid image 
data> said image search method comprising: 

an input stop of inputting a search query; 

a storage controlling step of storing a keyword table/ 
wherein the keyword table includes a keyword that 
corresponds to a plurality of images, and a plurality of 
levels of importance of the keyword with respect to said 
plurality of images; 

a searching step of searching said plurality of 
image data images according to the an input search query 
related to the keyword inputted by oaid input stop and 
keywords stored in relation to oaid image data ; 

an acquiring step of acquiring said plurality of 
levels of importance of the keywords keywor d / - the ■ 
importance being stored in relation to the keywords stored 
in relation to said image data based on the images searched 
by said searching step; and 

an output step of outputting the said plurality of 
imago data images searched by said searching step in an 
order according to said plurality of levels of importance 
of the keywords keyword acquired by said acquiring step. 

8. (Currently amended) An image search method 
according to claim 7, wherein the said plurality of image 
data images are output according to a standard level of 
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importance of a default when said plurality of levels of 
importance is are not stored in relation to the kcywordo 
keyword . 

9. (Currently amended) An image search method 
according to claim 7 , applied to an image search apparatus/ 
which searches image data according to keywords stored in 
relation to said image data, said image search method 
comprising: 

a storage controlling step of storing/ in storage 
moans / the keywords stored in relation to said imago data/ 
synonyms of said keywords/ and accordance between said 
keywords and said synonyms; 

an input step of inputting a search query; 

a searching stop of searching said image data 
according to the search query inputted by said input stcp> 
the keywords stored in relation to said imago data, and 
the synonyms stored by said storage controlling step; 

a first - acquiring step of acquiring importance of 
the keywords , the importance being stored in relation to 
the keywords stored in relation to said image data searched 
by said searching step; 

wherein a second acquiring stop of acquiring said 
a level of accordance between at least one synonym of said 
keyword and said input search query is acquired when said 
image data have boon searched according to said synonyms 
by said searching step; and 

an output step of outputting the said plurality of 
image data images are output, said plurality of images 
searched by said searching step in an order according to 
said plurality of levels of importance of the keywords 
keyword acquired by said first acquiring step and said level 
of accordance acquired by said second acquiring step. 
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10. (Cancelled) 

11 , (Cancelled) 

12 . (Currently amended) An image search method 
according to any of olaimo claim 7 through 9 , wherein said 
an input step comprises receiving a search query inputted 
from a WWW browser of a client of an information retrieval 
system on the Internet, said output step outputting the 
said plurality of imago data images searched by said 
searching step in a predetermined format, which includes 
HTML and XML, to said client in said order. 

13. (Currently amended) A storage medium that can 
be read by a computer and stores a program for executing 
an image search applied to an image search apparatus , which 
ooarcheo image data according to keywords otored in 
relation to said imago data; oaid storage medium comprising 
instructions for: 

inputting a - search quory; 

storing a keyword table, wherein the keyword table 
includes a keyword that corresponds to a plurality of images, 
and a plurality of levels of importance of the keyword with 
respect to said plurality of images; 

searching said plurality of imago data images 
according to the an input search query related to the 
keyword inputted by oaid input inotruction and the koywordo 
otored in relation to oaid imago data ; 

acquiring said plurality of levels of importance of 
the keywords keywor d s the importance being otored in 
relation to the koywordo otored in relation to oaid image 
data based on the images searched by said searching 
instruction; and 

outputting the said plurality of imago data images 
searched by said searching instruction in an order 



according to said plurality of levels of importance of the 
keywords keyword acquired by said acquiring instruction. 

14 . (Currently amended) A storage medium according 
to claim 13, wherein fefee said plurality of image data images 
are output according to a standard level of importance of 
a default when said plurality of levels of importance 
are not stored in relation to the keywords keyword , 

15. (Currently amended) A storage medium that can 
bo read by a computer and stores a program for executing 
an imago search applied to an image search apparatus 
according to claim 13 , which searches imago data according 
to keywords stored in relation to said imago data/ said 
storage medium comprising instructions for - : 

storing the keywords stored in relation to said image 
data/ synonyms of said keywords/ and accordance between 
said keywords and said synonyms; 

inputting a search query; 
— searching said imago data according to the search 
query inputted by said input instruction/ the keywords 
stored ■ in relation to said image data/ and the synonyms 
stored by said storing instruction; 

— acquiring importance of the keywords/ — the 
■ importance being stored in relation to the keywords stored 
in - rclation to said imago data searched by said searching 
instruction; 

wherein acquiring said a level of accordance between 
at least one synonym of said keyword and said input search 
query is acquired when said image data have been searched 
according to said synonyms by said searching instruction; 
and 

outputting the said plurality of image data images 
are output, said plurality of images searched by said 
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searching instruction in an order according to said 
plurality of levels of importance of the keywords keyword 
and said level of accordance. 

16. (Cancelled) 

17. (Cancelled) 

18 . (Currently amended) A storage medium according 
to any of claims claim 13 through 15 / wherein said an input 
instruction comprises receiving a search query inputted 
from a WWW browser of a client of an information retrieval 
system on the Internet, said output instruction outputting 
the said plurality of image data images searched by said 
searching instruction in a predetermined format, which 
includes HTML and XML, to said client in said order. 
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TITLE OF THE INVENTION 
IMAGE MANAGEMENT AND SEARCH APPARATUS, IMAGE MANAGEMENT 
AND SEARCH METHOD AND STORAGE MEDIUM 

BACKGROUND OF THE INVENTION 

Field of the Invention 

The present invention relates generally to an image management and search 
apparatus, an image manag e m e nt and search method and a storage medium, and more 
particularly to an image manag e m e nt and search apparatus, and an image managem e nt 
and search method that are suitable for managing and searching images by assigning 
keywords representing the images, and a storage medium storing a program for executing 
the image control and search method. 

Description of Related Art 

There is a conventional image database management technique of storing image 
data in a database and searching the stored image data. In the conventional image 
database management, keywords are assigned to images, and desired images can be 
searched for depending on the accordance of the keywords with search terms. According 
to an image searching process using this technique, even if a search term inputted by an 
operator does not completely match a keyword, the search term is made to match the 
keyword by replacing the search term with a term similar to the search term in notation, a 
synonym and a narrower term. This reduces the possibility that the image is not searched 
out due to the discord between the keyword and the search term. 

The above conventional technique, however, has a problem as described below. 
In the conventional image database management, it is almost impossible for all images to 
be represented by keywords. Some keywords represent predominant objects in images, 
and some keywords represent minor objects that appear in images but do not form 
subjects of the images. If the images are searched for by the keywords in such image 
management system, all the images to which a keyword matching the inputted search 
term is assigned are searched out. More specifically, an image whose subject is the 
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keyword and an image which appears as a small image at the edge of an image are 
searched out. Therefore, it takes a long time for a searcher to reach a desired image. 

SUMMARY OF THE INVENTION 
It is an object of the present invention to provide an image managem e nt and 
search apparatus, and an image manag e ment and search method which make it easier for 
a user to reach an image desired to be searched, by rearranging images to which 
keywords matching an inputted search term are assigned, according to the importance of 
the keywords to the image desired to be searched, and a storage medium storing a 
program for executing the image control and search method 

To attain the above object, according to a first aspect of the present invention, 
there is provided an image management and search apparatus, which searches image data 
according to keywords assigned to the image data, the image management and search 
apparatus comprising input means for inputting search terms, storage means storing the 
keywords assigned to the image data correspondingly to importance, search means for 
searching the storage means for keywords corresponding to images to be searched 
according to the search terms inputted by the input means, and rearranging means for 
rearranging images corresponding to the keywords searched by the search means 
according to the importance. 

Preferably, in the image management and search apparatus according to the first 
aspect, when the rearranging means rearranges the images corresponding to the keywords 
searched by the search means according to the importance, the rearranging means regards 
standard importance of a default as priority in a case where the importance is not stored 
in the storage means, and the rearranging means regards the importance as priority in a 
case where the importance is stored in the storage means, degree of the importance being 
determined according to whether or not the searched keywords represent subjects of 
images. 

To attain the above object, according to the second aspect of the present 
invention, there is also provided an image manag e m e nt and search method applied to an 
image managem e nt and search apparatus, which searches image data according to 
keywords assigned to the image data, the image management and search method 
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comprising an input step of inputting search terms, a storage controlling step of storing, 
in storage means, the keywords assigned to the image data correspondingly to 
importance, a searching step of searching the storage means for keywords corresponding 
to images to be searched according to the search terms inputted by the input steps, and a 
rearranging step of rearranging images corresponding to the keywords searched by the 
searching step according to the importance. 

Preferably, in the image management and search method according to the first 
aspect, when the images corresponding to the keywords searched by the search means are 
rearranged according to the importance, the rearranging step comprises regarding 
standard importance of a default as priority in a case where the importance is not stored 
in the storage means, and regarding the importance as priority in a case where the 
importance is stored in the storage means, degree of the importance being determined 
according to whether or not the searched keywords represent subjects of images. 

To attain the above object, according to the second aspect of the present 
invention, there is also provided a storage medium that can be read by a computer and 
stores a program for executing an image management and search method applied to an 
image manag e m e nt and search apparatus, which searches image data according to 
keywords assigned to the image data, the image manag e m e nt and search method 
comprising an input step of inputting search terms, a storage controlling step of storing, 
in storage means, the keywords assigned to the image data correspondingly to 
importance, a searching step of searching the storage means for keywords corresponding 
to images to be searched according to the search terms inputted by the input step, and a 
rearranging step of rearranging images corresponding to the keywords searched by the 
searching step according to the importance. 

To attain the above object, according to a second aspect of the present invention, 
there is provided an image manag e m e nt and search apparatus, which searches image data 
according to keywords assigned to the image data, the image manag e m e nt and search 
apparatus comprising input means for inputting search terms, storage means storing the 
keywords assigned to the image data correspondingly to importance, search means for 
searching the storage means for keywords corresponding to images to be searched or 
keywords closely related in meaning to the images to be searched according to the search 
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terms inputted by the input means, and rearranging means for rearranging images 
corresponding to the keywords searched by the search means according to the importance 
and closeness in meaning of the closely related keywords. 

Preferably, in the image manag e m e nt and search apparatus according to the 
second aspect, when the rearranging means rearranges the images corresponding to the 
keywords searched by the search means according to the importance and the closeness in 
meaning, the rearranging means calculates priority with reference to a relevant column of 
accordance in a line with no importance in a preset priority formula matrix in a case 
where the importance is not stored in the storage means; and the rearranging means 
calculates priority with reference to a relevant column of accordance in a line with 
relevant importance in the priority formula matrix in a case where the importance is 
stored in the storage means, degree of the importance being determined according to 
whether or not the searched keywords represent subjects of images. 

More preferably, in the image management and search apparatus according to the 
second aspect, the priority formula matrix is constructed in a manner such that the 
importance and the accordance are arranged correspondingly to one other in a form of a 
matrix, the rearranging means regarding the importance plus the accordance as priority if 
the accordance is not less than a predetermined threshold level, and regarding the 
accordance as priority if the accordance is less than the threshold level. 

To attain the above object, according to the second aspect of the present 
invention, there is also provided an image management and search method applied to an 
image management and search apparatus, which searches image data according to 
keywords assigned to the image data, the image management and search method 
comprising an input step of inputting search terms, a storage controlling step of storing, 
in storage means, the keywords assigned to the image data correspondingly to 
importance, a searching step of searching the storage means for keywords corresponding 
to the images to be searched or keywords closely related in meaning to the images to be 
searched according to the search terms inputted by the input step, and a rearranging step 
of rearranging images corresponding to the keywords searched by the searching step 
according to the importance and closeness in meaning of the closely related keywords. 

pPreferably, in the image manag e ment and search method according to the second 
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aspect, when the images corresponding to the keywords searched by the searching step 
are rearranged according to the importance and the closeness in meaning, the rearranging 
step comprises calculating priority with reference to a relevant column of accordance in a 
line with no importance in a preset priority formula matrix in a case where the 
importance is not stored in the storage means, and calculating priority with reference to a 
relevant column-of accordance in a line with relevant importance in the priority formula 
matrix in a case where the importance is stored in the storage means, degree of the 
importance being determined according to whether or not the searched keywords 
represent subjects of images. 

More preferably, in the image manag e ment and search method according to the 
second aspect, the priority formula matrix is constructed in a manner such that the 
importance and the accordance are arranged correspondingly to one other in a form of a 
matrix, the rearranging step comprising regarding the importance plus the accordance as 
priority if the accordance is not less than a predetermined threshold level, and regarding 
the accordance as priority if the accordance is less than said threshold level. 

To attain the above object, according to the second aspect of the present 
invention, there is provided a storage medium that can be read by a computer and stores a 
program for executing an image manag e ment and search method applied to an image 
manag e ment and search apparatus, which searches image data according to keywords 
assigned to the image data, the image managem e nt and search method comprising an 
input step of inputting search terms, a storage controlling step of storing, in storage 
means, the keywords assigned to the image data correspondingly to importance, a 
searching step of searching the storage means for keywords corresponding to images to 
be searched or keywords closely related in meaning to the images to be searched 
according to the search terms inputted by the input step, and a rearranging step of 
rearranging images corresponding to the keywords searched by the searching step 
according to the importance and closeness in meaning of the closely related keywords. 

In a preferred application of the present invention, the image managem e nt and 
search apparatus according to the first and second aspects is provided in a server of an 
information retrieval system on the Internet, the input means receiving search terms 
inputted by a client of the information retrieval system on the Internet, the search means 
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searching the storage means for keywords corresponding to images to be searched 
according to the received search terms, the rearranging means rearranging images 
corresponding to keywords searched by the search means according to the importance, 
and the image managem e nt and search apparatus returning the rearranged images in a 
predetermined format such as HTML and XML to the client. 

In a preferred application of the present invention, in the image managem e nt and 
search method according to the first and second aspects, the input step comprises 
receiving search terms inputted by a client of an information retrieval system on the 
Internet, the searching step comprising searching the storage means for keywords 
corresponding to images to be searched according to the received search terms, the 
rearranging step comprising rearranging images corresponding to keywords searched by 
the searching step according to the importance, and the rearranged images being returned 
in a predetermined format such as HTML and XML to the client. 

According to the image management and search apparatus, the image 
manag e ment and search method, and the storage medium of the first aspect of the present 
invention, keywords corresponding to images to be searched are searched for according 
to the inputted search terms, and images corresponding to the keywords searched are 
rearranged according to importance set for the keywords. More specifically, keywords 
representing objects forming subjects of images are determined to have much importance, 
and unimportant keywords representing minor objects in images, which do not form 
subjects of the images, are determined to have little importance. The images are stored 
together with the keywords for which the importance has been determined as mentioned 
above. The images to which the keywords matching the inputted search term are 
rearranged in the order of the importance. This makes it easy for the user to quickly 
reach a desired image. 

According to the image manag e m e nt and search apparatus, the image 
manag e m e nt and search method, and the storage medium of the second aspect of the 
present invention, keywords corresponding to images to be searched or keywords closely 
related in meaning to the images to be searched are searched for according to the inputted 
search terms, and images corresponding to the keywords searched are rearranged 
according to importance set for the keywords and closeness in meaning of the closely 
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related keywords. More specifically, keywords representing objects forming subjects of 
images are determined to have much importance, and unimportant keywords representing 
minor objects in images, which do not form subjects of the images, are determined to 
have little importance. Accordance (closeness in meaning) is determined according to the 
similarity of a keyword, which matches an inputted search term, with respect to the 
search term. The images are stored together with the keywords for which the importance 
and the accordance have been determined as mentioned above. The images to which the 
keywords matching the inputted search term are rearranged in the order of the importance 
and the accordance. This makes it easy for the user to quickly reach a desired image. 

BRIEF DESCRIPTION OF THE DRAWINGS 
The nature of this invention, as well as other objects and advantages thereof, will 
be explained in the following with reference to the accompanying drawings, in which like 
reference characters designate the same or similar parts throughout the figures and 
wherein: 

Fig. 1 is a schematic block diagram showing the structure of an image 
manag e m e nt and search apparatus according to first and second embodiments of the 
present invention; 

Fig. 2 is a flow chart showing the procedure of a control program stored in a 
ROM of the image managem e nt and search apparatus in Fig. 1 according to the first 
embodiment of the present invention; 

Fig. 3 is a flow chart showing the procedure of the control program stored in the 

ROM; 

Fig. 4 is a view showing a part of a table of image management information 
stored in a secondary storage device in Fig. 1 according to the first embodiment; 

Fig. 5 is a view showing a part of the table of image management information 
stored in the secondary storage device; 

Fig. 6 is a view of data on results of processing according to the procedure of Fig. 

2; 

Fig. 7 is a view of data on results of processing according to the procedure of Fig. 

2; 
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Fig. 8 is a flow chart showing the procedure of a control program stored in the 
ROM of the image manag e m e nt and search apparatus in Fig. 1 according to the second 
embodiment of the present invention; 

Fig. 9 is a flow chart showing the procedure of the control program stored in the 

ROM; 

Fig. 10 is a view showing a part of a table of image management information 
stored in a secondary storage device in Fig. 1 according to the second embodiment; 

Fig. 1 1 is a view showing a part of the table of image management information 
stored in the secondary storage device; 

Fig. 12 is a view showing a part of the table of image management information 
stored in the secondary storage device; 

Fig. 13 is a view of data on results of processing according to the procedure of 

Fig. 8; 

Fig. 14 is a view of data on results of processing according to the procedure of 

Fig. 9; 

Fig. 15 is a view of data on results of processing according to the procedure of 

Fig. 9; 

Fig. 16 is a view showing the storage contents of a storage medium that contains 
programs and related data for executing an image manag e m e nt and search method of the 
present invention; and 

Fig. 17 is a view describing a manner that the programs and the related data for 
executing the image management and search method of the present invention are supplied 
from the storage medium to the apparatus. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Preferred embodiments of the present invention will be described hereinbelow 
with reference to the accompanying drawings. 

Fig. 1 is a schematic block diagram showing the structure of an image 
management and search apparatus according to a first embodiment of the present 
invention. The image management and search apparatus of the first embodiment is 
comprised of a CPU 1, a display 2 5 a command input device 3, a ROM 4, a RAM 5 ? a 
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secondary storage device 6, and a storage medium reading device 7. The structure in Fig. 
1 is only an example, and the present invention is not limited to this particular example. 

The CPU 1 controls the whole image manag e m e nt and search apparatus of the 
present embodiment. The CPU 1 is connected to the display 2, the command input 
device 3, the ROM 4, the RAM 5, the secondary storage device 6, and the storage 
medium reading device 7 through a bus 8 so that data can be transmitted between the 
CPU 1 and the devices connected thereto. The CPU 1 executes the processing illustrated 
in the flow charts of Figs. 2 and 3 (the first embodiment) and Figs. 8 and 9 (the second 
embodiment), which will be described later. The display 2 is comprised of a CRT, a 
liquid crystal or the like, and displays images, searched results, and a keyword input 
column. The command input device 3 is comprised of a keyboard, and a mouse or the 
like, and is used to input instructions about a variety of processing operations, movement 
of a focused point, selecting an item, inputting characters and marks, and the like. The 
ROM 4 is a read only memory that contains a variety of processing programs to be 
executed by the CPU 1 . A program storage part 4a, which is a part of the ROM 4, 
contains a keyword search program (control program) and data according to the present 
invention. 

The RAM 5 is a read and write enable memory that temporarily contains the 
results of various operations and inputted information. The secondary storage device 6 is 
comprised of a hard disk, an MO (Magneto Optic Disk), or the like. The secondary 
storage device 6 contains images, keywords and importance information assigned to the 
images, and image data (the first embodiment), images, keywords and importance 
information assigned to the images, a table used to search for search terms by 
development of the search terms into synonyms therefor, and image data (the second 
embodiment), and the like. The storage medium reading device 7 is comprised of a CD- 
ROM drive, an FD (floppy disk), or the like, and is used to read programs from storage 
mediums (i.e., a CD-ROM 71 and a floppy disk 72) that contain a program code means. 
The program of the present invention is stored in storage mediums such as an FD, a CD- 
ROM, a ROM, and a magnetic tape. The program may be read from the storage medium 
reading device 7 and be stored in the RAM 5. 

Figs. 4 and 5 show a part of an image management information table stored in the 
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secondary storage device 6 of the image manag e m e nt and search apparatus according to 
the first embodiment of the present invention. Fig. 4 shows an image information table 
wherein each image is provided with an image ID as an image identification number in 
correspondence with each image file name. Fig. 5 shows a keyword table as a 
correspondence table of image IDs and keywords. The keyword table is provided with 
importance fields, and the importance field of each record contains the importance "1" 
through "3." The importance is predetermined so that a keyword representing an object 
forming a subject of an image can be of much importance, and that an unimportant 
keyword such as a keyword representing a minor object, which does not form a subject of 
an image, can be of little importance. In the case where there is already a large amount of 
data in the database and it is expensive to newly assign the importance, no data may be 
stored in the importance fields. The above-mentioned range and possible values of the 
importance are only one example of the present embodiment. 

Fig. 17 is a view describing how programs and related data for executing the 
image management and search method of the present invention are supplied from the 
storage medium to a device. The programs and the related data for executing the image 
managem e nt and search method of the present invention are supplied to a device 1 72 by 
inserting a storage medium 171 such as a floppy disk and a CD-ROM into a storage 
medium drive insertion opening 172 formed in the device 1 72. Then, the program and 
the related data are installed onto a hard disk from the storage medium 171, and they are 
loaded into the RAM from the hard disk. Alternatively, the program and the related data 
are directly loaded into the RAM without being installed on the hard disk. This enables 
the execution of the program and the related data. 

To execute a program for executing the image management and search method of 
the present invention by the image manag e m e nt and search apparatus according to the 
first embodiment or the later-described second embodiment of the present invention, the 
program and the related data are supplied to the image manag e ment and search apparatus 
in the procedure described with reference to Fig. 1 7, or the program and the related data 
are stored in the image manag e ment and search apparatus in advance. This enables the 
execution of the program. 

Fig. 16 is a view showing an example of the structure of storage contents of a 
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storage medium that contains the program and the related data for executing the image 
management and search method of the present invention. The storage medium of the 
present invention contains, for example, volume information 161, directory information 
162, a program execution file 163, and a program/related data file 164. The program for 
executing the image management and search method of the present invention is coded 
according to the flow charts of Figs. 2 and 3 (the first embodiment) or the flow charts of 
Figs. 8 and 9 (the second embodiment). 

Referring next to the flow charts of Figs. 2 and 3 and Figs. 5 through 7, a detailed 
description will be given of a control process of the image manag e m e nt and search 
apparatus according to the present invention that is constructed in the above-mentioned 
manner. The CPU 1 executes this control process. 

Fig. 2 is the flow chart showing the procedure of a control program stored in the 
program storage part 4a of the ROM 4 of the image management and search apparatus. 
Turning on a power supply of the system starts the execution of a step S 1 to perform a 
necessary initialization process. The occurrence of an event is waited for in a step S2. 
When an event for executing a command occurs due to an operation of the command 
input device 3, the process goes to a step S3. If the event is an image searching event, the 
process goes to a step S5. If the event is not the image searching event, the process goes 
to a step S4 to process the event. In the step S5, the keyword table in Fig. 5 is searched in 
a query language according to searching conditions to thereby extract relevant records. 
Fig. 6 shows an example of the extracted records. Then, the process goes to a step S6 to 
determine the priority. 

Fig. 3 is the flow chart showing the details of a priority calculating process in the 
step S6 in Fig. 2. In a step S21, a loop is started for the records extracted in the step S5. 
In a step S22, the importance field of each record is evaluated. If there is no importance 
data, the process goes to a step S23, and if there is the importance data, the process goes 
to a step S24. In the step S23, a standard importance is regarded as the priority. The 
standard importance is a default value of the importance, which is used if the importance 
has not yet been specified. In the present embodiment, "2" is regarded as the standard 
importance. In the step S24, the preset importance of the keywords for the images in the 
table in Fig. 5 is regarded as the priority. After the priority has thus been determined, the 
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process goes to a step S25 to determine whether the processing of the last record has been 
completed or not. If the processing of the last record has been completed, the priority 
calculating process is finished. After the priority is calculated in the above-mentioned 
manner, the process goes to a step S7 to rearrange the images according to the priority. 
Fig. 7 shows an example of the rearranged records. Then, the process goes to a step S8 to 
output the images through the display 2. 

As stated above, the image management and search apparatus according to the 
first embodiment of the present invention has the secondary storage device 6, which 
contains the keywords assigned to the image data correspondingly to the importance, and 
the CPU 1 , which searches the secondary storage device 6 for keywords corresponding to 
images to be searched according to an inputted search term and controls the apparatus in 
such a manner as to rearrange the searched images according to the importance. 
Therefore, the operation and effect of the image management and search apparatus 
according to the first embodiment of the present invention are as described hereinbelow. 

In this arrangement, the importance field of each record conforming to the 
searching conditions is evaluated, and if there is no importance data, the standard 
importance of a default is regarded as the priority. On the other hand, if there is 
importance data, the predetermined importance of a keyword corresponding to the image 
is regarded as the priority. After the priority of every image is found, the images are 
rearranged according to the priority and the rearranged images are outputted. More 
specifically, keywords representing objects forming subjects of images are determined to 
have much importance, and unimportant keywords representing minor objects in images, 
which do not form subjects of the images, are determined to have little importance. The 
images are stored together with the keywords for which the importance has been 
determined as mentioned above. Therefore, the images to which the keywords matching 
the inputted search term are rearranged according to the importance. This makes it easy 
for the user to quickly reach a desired image. 

There will now be described an image manag e m e nt and search apparatus 
according to a second embodiment of the present invention. 

As shown in Fig. 1 , the image management and search apparatus of the second 
embodiment is comprised of the CPU 1, the display 2, the command input device 3, the 
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ROM 4, the RAM 5, the secondary storage device 6, and the storage medium reading 
device 7 as is the case with the first embodiment. Each part of the image manag e m e nt 
and search apparatus has the same structure as in the first embodiment, and therefore, it 
will not be described hereinbelow. 

Figs. 10 through 12 are views showing a part of an image management 
information table stored in the secondary storage device 6 of the image management and 
search apparatus according to the second embodiment. Fig. 10 shows an image 
information table wherein an image ID as an image identification number is assigned to 
each image corresponding to each image file name. Fig. 1 1 shows a keyword table 
wherein image IDs and keywords are arranged correspondingly to one another. This 
table has importance fields, and the importance from "1" through "3", which is set 
according to the same standard as that of the first embodiment, is stored in the importance 
field of each record. If there is already a large amount of data in the database and it is 
expensive to newly assign the importance, no data is stored in this importance fields. Fig. 
12 shows a synonym-search term table wherein words having nearly the same meaning as 
an inputted search term are stored as synonyms. This synonym-search term table has 
accordance fields, and the accordance "0" through "1" indicating the conceptual distance 
between the search term and the synonyms (the accordance "1" indicates the same 
concept) is stored in the accordance field of each record. The range of the importance 
and the accordance is only an example of the present embodiment, and the present 
invention is not limited to this. The accordance can be determined in a variety of 
methods such as a manual method and an automatic generating method using electronic 
dictionary data or the like. 

Referring next to the flow charts of Figs. 8 and 9 and Figs. 11 through Fig. 15, a 
detailed description will be given of the control process of the image manag e ment and 
search apparatus according to the second embodiment that is constructed in the above- 
mentioned manner. The CPU 1 executes this control process. 

Fig. 8 is the flow chart showing the procedure of a control program stored in the 
program storage part 4a of the ROM 4 in the image manag e ment and search apparatus. 
When the power supply of the system is turned on, a step S81 is executed to perform a 
necessary initialization. The occurrence of an event is waited for in a step S82. When an 
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event for executing a command occurs due to an operation through the command input 
device, the process goes to a step S82. If the event is an image searching event, the 
process goes to a step S85. If the event is not the image searching event, the process goes 
to a step S84 to process the event. In the step S85, an inputted search term is developed 
into synonyms with reference to a synonym-keyword development table (Fig. 12) to find 
the accordance. For example, if a search term "motorbike" is developed, a synonym 
"motorcycle" is searched out and the accordance is 0.6. If a search term "motorcycle" is 
inputted, the accordance is 1 since the search term "motorcycle" accords with the 
synonym "motorcycle". Then, the process goes to a step S86 to search the keyword table 
in Fig. 1 1 by the words developed in the table of Fig. 12 and extract relevant records. 
Fig. 13 shows an example of the extracted records. Then, the process goes to a step S87 
to calculate the priority. 

Fig. 9 is the flow chart showing the details of the priority calculating process in 
the step S87 of Fig. 8. In a step S91, a loop is started for each record extracted in the step 
S86. In a step S92, the importance field of each extracted record is evaluated. If there is 
no importance data, the process goes to a step S93, and if there is importance data, the 
process goes to a step S94. In the step S93, the priority is calculated according to a 
priority formula matrix with reference to a column of the relevant accordance in a line 
with no importance. In the step S94, the priority is calculated according to the priority 
formula matrix with reference to a column of the relevant accordance in a line with the 
relevant importance. Fig. 14 shows an example of the priority formula matrix. If the 
accordance found in the step S85 is not less than the accordance of a predetermined 
threshold level (0.5), the importance plus the accordance is regarded as the priority. If the 
accordance found in the step S85 is less than the threshold level, the accordance is 
regarded as the priority. The formula of the priority, the formula of the priority formula 
matrix and the fineness of segmentation are determined according to the range 
distribution of the importance and the accordance. 

After the calculation of the priority as described above, the process goes to a step 
S95 to determine whether the process with respect to the last record is completed or not. 
If completed, the process goes to a step S96, and if not completed, the process returns to 
the step S91 . The keywords are grouped per image in the step S96. If a plurality of 
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keywords correspond to one image in a step S97, the maximum value of the priority 
found in the step S93 or S94 is regarded as the priority of the image. Fig. 15 shows a 
search result with respect to a plurality of records corresponding to the same image. 
After the priority is found for each image as described above, the process goes to a step 
S88 in Fig. 8 to rearrange the images according to the priority. Then, the process goes to 
a step S89 to output the search result through the display 2. 

As stated above, the image manag e ment and search apparatus according to the 
second embodiment has the secondary storage device 6 that contains the keywords 
assigned to the image data correspondingly to the importance, and the CPU 1 that 
controls the apparatus in such a manner as to search the secondary storage device 6 for 
keywords corresponding to an inputted search term and keywords having the same or a 
close meaning as or to the search term and rearrange images, to which the searched 
keywords are assigned, according to the importance and the closeness in meaning. 
Therefore, the operation and effect of the image management and search apparatus 
according to the second embodiment are as described hereinbelow. 

In this arrangement, the importance field of each record conforming to the 
searching conditions is evaluated, and if there is no importance data, the priority is 
calculated according to the priority formula matrix with reference to a relevant column of 
accordance in a line with no importance. On the other hand, if there is importance data, 
the priority is calculated according to the priority formula matrix with reference to a 
relevant column of accordance in a line with relevant importance. After the priority of 
every image is found, the images are rearranged according to the priority and the 
rearranged images are outputted. More specifically, keywords representing objects 
forming subjects of images are determined to have much importance, and unimportant 
keywords representing minor objects in images, which do not form subjects of the 
images, are determined to have little importance. The accordance (the closeness in 
meaning) is determined according to the similarity of a keyword, which matches an 
inputted search term, with respect to the search term. The images are stored together with 
the keywords for which the importance and the accordance have been determined as 
mentioned above. Therefore, the images to which the keywords matching the inputted 
search term are rearranged according to the importance and the accordance. This makes 
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it easy for the user to quickly reach a desired image. 

In the first and second embodiments of the present invention described above, the 
present invention is applied to the simplex image management and search apparatus; 
however, the present invention is not limited to this. For example, the present invention 
may also be applied to a system wherein an image management and search apparatus 
connects to external devices (e.g., a computer and a printer) through a network such as 
the Internet and a LAN (Local Area Network). 

In the first and second embodiments of the present invention described above, the 
search result is outputted through the display 2 of the image manag e m e nt and search 
apparatus; however, the present invention is not limited to this. For example, the image 
manag e ment and search apparatus may be connected to a printer that outputs the search 
result. 

In the first and second embodiments of the present invention described above, the 
keyword tables in Figs. 5 and 1 1 are given as examples; however, the present invention is 
not limited to the keywords and the importance shown in Fig. 5 and 11. The keywords 
and the importance may be determined arbitrarily. 

The present invention may be applied to either a system comprised of plural 
equipment or an apparatus comprised of single equipment. For example, the image 
management and search apparatus of the present invention may be provided on a server 
of an information retrieval system such as a WWW (World Wide Web) on the Internet. 
Then, a search term inputted from a WWW browser of a client is issued through the 
WWW, and the image management and search apparatus of the present invention 
provided on the WWW server receives the search term to search and rearrange images. 
The search result is shaped in the HTML (Hyper Text Markup Language) or the XML 
(Extensible Markup Language), and is then returned to the client. 

Moreover, it is needless to say that the present invention can also be achieved by 
supplying a program to the system or the apparatus. It is needless to say that the present 
invention can also be achieved by providing the system or the apparatus with a storage 
medium containing a program code of software realizing the functions of the above 
described embodiments, and reading a program code from the storage medium by a 
computer (or a CPU and an MPU) of the system or the apparatus. In this case, the 
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program according to the present invention, in other words, the program code read from 
the storage medium executes the functions of the above described embodiments. The 
storage medium containing the program code constitutes the present invention. 

The storage medium for providing the program code is 5 for example, a floppy 
disk, a hard disk, an optical disk, a magneto optic disk, a CD-ROM, a CD-R, a magnetic 
tape, a non-volatile memory card, and a ROM. 

Moreover, it is needless to say that the functions of the above described 
embodiments can be realized not only by executing the program code read from the 
computer, but also by executing a part or the whole of the actual processing by means of 
an OS or the like operating on the computer. 

Furthermore, it is also needless to say that the functions of the above described 
embodiments can be realized in such a manner that the program code read from the 
storage medium is written in a memory provided in a function expanding board inserted 
into the computer or a function expanding unit connected to a computer, and then a CPU 
or the like provided in the function expanding board or the function expanding unit 
executes a part or the whole of the actual processing according to instructions of the 
program code. 
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ABSTRACT OF THE DISCLOSURE 
There are provided an image manag e m e nt and search apparatus, and an image 
manag e m e nt and search method, which make it easier for a user to reach an image 
desired to be searched, by rearranging images to which keywords matching an inputted 
search term are assigned, according to the importance of the keywords to the image 
desired to be searched, and a storage medium storing a program for executing the image 
control and search method. Keywords corresponding to images to be searched are 
searched for according to the inputted search terms, and images corresponding to the 
keywords searched are rearranged according to importance set for the keywords. The 
images are stored together with the keywords for which the importance has thus been 
determined. The images to which the keywords matching the inputted search term are 
rearranged in the order of the importance. 
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